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ABSTRACT 



A supplemental information embedding apparatus having a 
device for capturing image data, a first memory for storing 
image data indicating the presence of copyright protection, 
a first encoder for encoding the image data obtained by the 
image capturing means to a first format digital image data by 
n stages of processing (n is a positive integer greater than or 
equal to 2), a second memory for storing the first format 
digital image data produced in the digital image data pro- 
ducing means, a second encoder for encoding the image data 
indicating the copyright data supplied from the image cap- 
turing device or the first memory to a second format digital 
watermark by i stages of processing (i is a positive integer 
less than n), a third memory for storing the second format 
digital watermark encoded by the second encoder, a decoder 
for decoding the first format digital image data stored in the 
second memory up to a desirable stage of decoding, a 
divider for dividing the second format digital watermark 
stored in the third memory into predetermined format 
blocks, an embedder for dispersively embedding the blocks 
of the digital watermark into the image data encoded up to 
i stage by the first encoder or the irate data decoded up to i 
stage by the decoder, a third encoder for encoding the image 
data obtained by the embedder by exerting operations on and 
after j stage (j«n-l) of encoding so as to produce the first 
format digital image data, and a fourth memory for storing 
the first format digital image data encoded by the third 
encoder. 

24 Claims, 10 Drawing Sheets 
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APPARATUS AND METHOD FOR 

EMBEDDING/UNEMBEDDING 
SUPPLEMENTAL INFORMATION 

FIELD OF THE INVENTION 

The present invention relates to image data processing 
techniques, such as JPEG systems, MPEG systems, etc., and 
more particularly to an apparatus and a method for 
embedding/unbedding supplemental information and a 
record medium therefor, which are applied with an elec- 
tronic watermark technology for embedding the copyright 
data or personal identification data into the digital image 
data. 

BACKGROUND OF THE INVENTION 

Recently, DCT (Discrete Cosine Transform)-based image 
compression/expansion technologies such as JPEG (Joint 
Photographic Coding Experts Group) systems and MPEG 
(Moving Picture Experts Group) systems have come into 
practice as fundamental technologies for realizing multime- 
dia. 

An electronic still camera has become widespread in use, 
which converts an imaging object into an electric signal by 
an optoelectric transducer device such as a CCD (charge 
coupled device), performs the specific digital signal 
processing, then records the digitized still rage into a record- 
ing medium such as a memory card. The image data are 
digitized for compressing the huge amount of its code, using 
the high efficiency encoding such as the JPEG (Joint Pho- 
tographic Coding Experts Group). 

Further, with the establishment of the high efficiency 
encoding technologies for such a still image, the digital 
processing of a moving image has been developed. The high 
efficiency encoding technology used in the moving picture is 
to encode the image data at the lower bit rate so as to 
improve the efficiency of the digital transmission or 
recording, and an MPEG-1 (ISO/IEC 11172) of the high 
efficiency encoding standard is already used in various 
media, such as a video CD (Compact Disc), a CD-I 
(Compact Disc -Interactive), etc. Further, an MPEG-2 
(Moving Picture Experts Group 2) standard which is a 
standard corresponding to the image quality of the conven- 
tional broadcasting has also been put to practical use, and 
picture software items based on the MPEG-2 standard have 
been on sale. 

An embodiment of the conventional JPEG system image 
compressing apparatus is shown in FIG. 1. 

An image that is input to an image compressor is divided 
into 8x8 pixel blocks, and then rearranged to frequency 
components by the DCT processing carried out in a DCT 
processor 51. The blocks rearranged to frequency compo- 
nents (DCT coefficients) are transformed into one dimen- 
sional format from two dimensional format by the zigzag 
scanning or the alternate scanning carried out in a scanner 
52. A quantizer 53 divides each DCT coefficient of the 
blocks converted into one dimensional format by a quanti- 
zation matrix table coefficient, and then obtains quantized 
data from the result of this division by rounding it to a 
nearest whole figure. A VLC processor 54 carries out the 
reversible compression (variable length encoding) of the 
quantized DCT coefficients. Compressed data of the blocks 
are thus obtained. By executing the above operations for all 
blocks of input images successively, compressed image data 
for one frame are obtained. 

Further, in the case of the MPEG to compress moving 
pictures, the DCT processed blocks are quantized and the 
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quantized blocks are transformed into one dimensional 
format from two dimensional format. That is, the scanning 
procedures of the MPEG system is reverse to those of the 
JPEG system. 

5 Now, the configuration of the JPEG system image decoder 
is partially shown in FIG. 2. 

A VLD processor 61 carries out the variable length 
decoding of input image compression data by looking up the 
Huffman table. The variable length decoded data (quantized 

10 DCT coefficients) are reverse quantized by multiplying by 
coefficients of the quantization matrix table in a reverse 
quantizer 62. The quantized DCT coefficients are reverse 
scan transformed from one dimensional format to two 
dimensional format blocks in a reverse scanner 63, and then 

is 

inverse DCT processed in an IDCT processor 64, and 8x8 
pixel blocks which are close to an original image are thus 
reproduced. 

Further, in the case of the MPEG, the variable length 

2Q decoded data are reverse quantized after being reverse scan 
transformed. Demands for and the necessity of embedding 
such supplemental information 72 as character information 
for explaining contents of an image 71 in an electronic 
album, as shown in FIG. 3, ID information for individual 

25 identification for the purpose of preventing illegal copying 
of digital images are increasing in recent years. 

For adding character information, there are several meth- 
ods available, including a method of storing image data and 
character information in separate files, and a method of 

30 producing one file by pasting image data and character 
information on separate application softwares. 

However, the method of storing image data and character 
information in separate files results in an increase of the 
number of files. In addition, the method of adding image 

35 data and character information on separate application soft- 
ware results in an increase in the file size and necessitates an 
exclusive reading device and a software when a file name 
changes. 

Further, recently, the need for protecting the copyrights of 

40 these digital still pictures or digital moving pictures has 
increased. One of the techniques for resolving these issues 
is digital watermark technology. 

The digital watermark technology embeds the digital 
watermark such as a sentence or a logo mark which could be 

45 used to protect the copyright or a trademark, into the image 
data of the digital still picture such as a JPEG compression 
image or the digital moving picture such as an MPEG 
compression image. The copyright data are embedded in the 
JPEG image data or MPEG image data in the way that the 

50 embedded data are not displayed in accordance with the 
JPEG or the MPEG standard, even if the JPEG image data 
or the MPEG image data are decoded. 

If a JPEG or an MPEG image data accompanied by a 

55 digital watermark representing a copyright has been illegally 
copied, a copyright owner can confirm his copyright by 
unbedding the digital watermark from the copy of image 
data in accordance with a prescribed procedure. 
Now, a conventional system for embedding the digital 

60 watermark into the JPEG image will be described. 
Conventionally, the digital watermark is embedded into the 
JPEG image after a JPEG compression has been made. 

That is, the image data captured by the image capturing 
device such as a CCD are divided into 8x8 pixel blocks and 

65 are input to the frequency converter for carrying out a DCT 
(Discrete Cosine Transform) which is for a quadrature 
transform encoding. In the DCT, which operates as a fre- 
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quency converter, the 8x8 pixel blocks are arranged in quality of reproduced image is reduced by a part of original 

different order of the frequency ingredients. The rearranged image data being lost by the supplemental information, 
blocks (DCT coefficient) are rounded off to the nearest 

integer by the quantizer, after the coefficient corresponding SUMMARY OF THE INVENTION 

to a matrix is divided by the quantization matrix table. 5 it is, therefore, an object of the present invention to 

Accordingly a first stage of compression processing (non- provide a supplemental information embedding apparatus 

reversible compression) is carried out. a nd a supplemental information embedding method capable 

The quantized matrix data are converted into one dimen- of embedding the digital watermark, which protects the 
sional format from two dimensional format by a scanner. copyright, to the captured digital image data at a real-time 
The data converted into one dimensional format are pro- 30 without using a large scale system, in the electronic still 
cessed through variable length encoding by looking up the camera or the digital video recorder, 
encoding table, such as a Huffman table for instance, and a second object of the present invention is to provide a 
further processed through the second compression process- supplemental information embedding/unbedding apparatus 
ing (reversible compression) by a VLC (Variable Length and its method capable of minimizing the reduction of 
enCoder) processor, so that the compressed data of the 8x8 15 reproduced image quality caused by embedding the supple- 
pixel blocks are obtained. mental information into the image compression data. 

After performing the operations mentioned above to all Another object of the present invention is to provide a 
blocks of input image, the ultimate compressed data are supplemental information embedding/unbedding apparatus 
obtained and stored in the external memory such as a and its method capable of minimizing a reduction of data 
memory card. 20 compression ratio resulting from the embedding of supple- 
Then, a digital watermark presenting a copyright, a logo mental data into image compression data, 
mark, or a trademark is embedded into the JPEG compres- a further object of the present invention is to provide an 
sion image data by using a special purpose machine, such as i^ge decoder and an image expansion method capable of 
a computer equipped with software for embedding digital completely eliminating the reduction of reproduced image 
watermarks, before the image data are stored in the external q Ualitv caused by embedding the supplemental information 
memory. mto tne j ma g e compression data. 

In the conventional system described above, there was a In order to achieve tne above objects> a supplemental 

need to embed the digital watermark, such as the copyright i n f orma tion embedding apparatus according to one aspect of 

data, to the JPEG image data by using a special purpose 3Q the present invention includes a device for capturing image 

software and computer for embedding the digital watermark data> a first memory for storing image data indicating the 

after the JPEG image data are obtamed by the JPEG image presence of copyright protection, a first encoder for encod- 

producing device such as an electronic still camera. As a mg the ^ge data obtained by the image capturing device 

result, the system became large. t0 a first format digital data by n stages of proce ssing 

Further, though the digital image data can be easily 35 ( n i s a positive integer greater than or equal to 2), a second 

produced with the wide spread of the electronic still camera memory for storing the first format digital image data 

and the portable digital video equipment, there are no encoded by the first encoder, a second encoder for encoding 

systems capable of embedding the copyright data in a the image data indicating the copyright data supplied from 

real-time manner at the time of image capturing operation. the image capturing device or the first memory to a second 

On the other hand, adding ID information for identifying 40 format digital watermark by i stages of processing (i is a 

individuals to image data can be performed through different positive integer less than n), a third memory for storing the 

methods, including a method of adding images such as signs second format digital watermark encoded by the second 

for identifying individuals in case of, e.g., image data in a encoder, a decoder for decoding the first format digital 

bitmap format, into the LSB (Least Significant Bit) in 8 bits image data stored in the second memory up to a desirable 

(256 tones) for expressing pixels, and a method of replacing 45 stage of decoding, a divider for dividing the second format 

DCT processed low frequency band image data, in case of digital watermark stored in the third memory into predeter- 

a JPEG or an MPEG image compression data, with spread mined format blocks, an embedder for dispersive ly embed- 

spectrum ID information. ding the blocks of the digital watermark into the image data 

However, when embedding ID information for identify- encoded up to i stage by the first encoder or the image data 

ing individuals into image data, the quality of the reproduced 50 decoded up to i stage by the decoder, a third encoder for 

image is reduced by the image data being partially lost by encoding the image data obtained by the embedder by 

the supplemental information. exerting operations on and after j stage (j=n-l) of encoding 

As described above, though the digital image data are so as to produce the first format digital image data, and a 

easily produced with the popularity of the electronic still fourth memory for storing the first format digital image data 

camera or a portable digital video, the conventional system 55 encoded by the third encoder. 

had a drawback in that it can not embed the copyright data A supplemental information embedding apparatus accord- 
in a real-time manner during image capturing, so that it had ing to second aspect of the present invention includes an 
to embed the digital watermark, such as the copyright, by orthogonal transformer for orthogonally transforming 
using a large scale system using software or a computer for divided blocks of image data, a quantizer for quantizing the 
embedding the digital watermark. go divided blocks of image data after the orthogonal transform, 
As described above, the conventional method of embed- and an embedder for embedding supplemental information 
ding supplemental information such as character informa- in place of at least a part of a zero-run section immediately 
tion into image compression data also has a drawback of preceding an EOB (End of Block) in each of the quantized 
increasing in the number of files and the size of files. blocks of image data. 

Furthermore, in a conventional method of embedding 65 A supplemental information embedding apparatus accord- 
supplemental information such as ID information for iden- ing to another aspect of the present invention includes an 
tifying individuals, etc. into image compression data, the orthogonal transformer for orthogonally transforming 
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divided blocks of image data, a quantizer for quantizing the FIG. 3 is a diagram for explaining character information 

divided blocks of image data after the orthogonal transform, that is embedded into image data. 

and an embedder for embedding supplemental information FIG. 4 is a block diagram showing the embodiment of the 

in place of the head of a zero-run section immediately sup pi em ental information embedding apparatus according 

preceding an EOB (End of Block) in each of the quantized 5 to the present invention; 

blocks of image data. r , 

Asupplemental information embedding apparatus accord- ™?\ 5 ls a b , lock dia f am S ^ 0W1D ? ao embodiment of the 
ing to still another aspect of the present invention includes dl S ltal data P roducer shown "> FIG. 4; 
an orthogonal transformer for orthogonally transforming FIG - 6 ls a flowchart showing the supplemental informa- 
divided blocks of image data, a quantizer for quantizing the 10 tion embedding method of the present invention; 
divided blocks of image data after the orthogonally FIG. 7 is a block diagram showing an embodiment of the 
transform, a divider for dividing supplemental information apparatus for unbedding the digital watermark; 
corresponding to one frame of the image data into a plurality FIG. 8 is a block diagram showing the configuration of an 
of blocks, and an embedder for dispersively embedding the image compressor of the present invention; 
divided blocks of supplemental information into the image 15 FIG. 9 is a flowchart showing the steps of image process- 
data in place of the head of a zero-run section immediately mg operation in the supplemental information embedding 
preceding an EOB (End of Blocks) of the quantized blocks apparatus, as shown in FIG. 8; 

of image data. Pjq * s a d^g,.^ showing examples of one dimen- 

An image decoder for decoding image compression data sional blocks after quantization by the image compressor 

wherein supplemental information is embedded in place of 20 shown in FIG $• 

at least a part of a zero-run section immediately preceding an FIG. 11 is a diagram showing the sections of one dimen- 

EOB (End of Block), which is obtained by dividing image siona , blocks after quantization mto which supp^^i 

data, orthogonal transforming the dmded blocks of image ^fo^o,, can be bedded, as shown in FIG. 10; 

data and quantizing the orthogonal transformed blocks of „ T ^, . t , . , 

image data, according to still another aspect of the present 25 .. FIG \ 12 f anagram ! showing examples of quantized one 

invention, includes an unbedder for unbedding the supple- dimensional blocks embedded with supplemental informa- 

mental information from the image compression data, a ' 

memory for storing the unbedded supplemental information, FIG * 13 is a block dia g ram showing the configuration of 

a replacer for replacing the supplemental information an ima e e decoder of the P rcse nt invention; and 

embedded in the image compression data with a zero-run, a 30 FIG - 14 is a flowchart showing the steps of supplemental 

reverse quantizer for reverse quantizing the zero-replaced information unbedding operation in the supplemental infor- 

image compression data, and a reverse orthogonal trans- mation unbedding apparatus, as shown in FIG. 13. 

former for reverse orthogonal transforming the reverse DESCRIPTION OF THE PREFERRED 

quantized image compression data. EMBODIMENTS 

An image decoder for decoding reversible compressed 35 r™ . . 4 . .„ . , , . , t .. . iL 

image data wherein supplemental information is embedded ™ c ^ u ,1 " * 

in place of at least a part of a zero-run section immediately rc *™f * th ?F lGS u * U ' . u 

preceding an EOB (End of Block) obtained by dividing FIG f * and * m blo 1 ck f a g rams showing the embodi- 

image data, orthogonal transforming the divided blocks of ment ?. f the ^PPkmcntal information embedding apparatus 

image data and quantizing the orthogonal transformed 40 ^rding to the present invention. FIG. 6 is a flowchart 

blocks of image data, according to still another aspect of the showin g. the operation of the apparatus or a supplemental 

present invention, includes a reversible decoder for revers- ^formation embedding method. 

ible decoding the image compression data, an unbedder for In this embodiment, the digital image data will be 

unbedding the supplemental information from the reversible explained by taking the JPEG which is the still image 

decoded image compression data, a memory for storing the 45 com P ression standard as an example, 

unbedded supplemental information, a replacer for replacing ,a F1G - 4 > a terminal a of a switch 107 is connected to a 

the supplemental information embedded in the image com- CCD 1°* for capturing an image by optoelectrically trans- 

pression data with a zero-run, a reverse quantizer for reverse during an image object light from an image object, a 

quantizing the zero-replaced image compression data, and a terminal cl and a terminal c2 are connected to a digital 

reverse orthogonal transformer for reverse orthogonal trans- so image data processor 102 for encoding the digital image data 

forming the reverse quantized image compression data. aaa< producing the compressed data such as the JPEG, and a 

Additional objects and advantages of the present inven- terminal a is connected to a digital watermark processor 104 

tion will be apparent to persons skilled in the art from a study for carr ying out conversion of the digital watermark such as 

of the following description and the accompanying the copyright data into a reduced-size picture, producing the 

drawings, which are hereby incorporated in and constitute a 55 compressed data such as the JPEG, and writing in and 

part of this specification. reading out of a digital watermark memory 105. 

RRTFF DFSCRIPTTON OF THF AWTNfK ^ ^ P rOCeSSOr 102 15 coimect ed 

DESCRIPTION OF THE DRAWINGS to a terminal f of a switch 108, a terminal g of the switch 108 

A more complete appreciation of the present invention is connected to an external memory 103 such as the memory 

and many of the attendant advantages thereof will be readily 60 card for storing the compressed data or a terminal f of a 

obtained as the same becomes better understood by refer- switch 109, and the digital watermark processor 104 is 

ence to the following detailed description when considered connected to a terminal i of the switch 109. 

in connection with the accompanying drawings, wherein: Then, the switches 107, 108, and 109 are respectively 

FIG. 1 is a block diagram showing the configuration of a controlled by a mode designator 106. 

conventional JPEG system image compressor; 6 5 On the other hand, FIG. 5 is a block diagram showing an 

FIG. 2 is a block diagram showing the configuration of a embodiment of the digital image data producer, as shown in 

conventional JPEG system image decoder; FIG. 4. 
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In FIG. 5, a terminal cl is connected to a DCT processor 

201 for carrying out the quadrature transform encoding by 
dividing the image data into 8x8 pixel blocks. The DCT 
processor 201 is connected to a quantizer 202. The quantizer 

202 is then connected to a scanner 203 for scanning to 5 
convert the matrix data from the two dimensional format to 
the one dimensional format. 

Further, a terminal c2 is connected to an information 
embedder 204, a terminal f is connected to a VLC processor 
205 and a terminal q of a switch 208, and a terminal r of the 30 
switch 208 is connected to the information embedder 204 
via a VLD (variable length decoder) processor 206. 

Furthermore, a terminal p of a switch 207 is connected to 
the scanner 203, a terminal n is connected to the VLC 
processor 205 and the information embedder 204, and a 15 
terminal o is connected to the information embedder 204. 

Then, the switches 207 and 208 are respectively con- 
trolled by the mode designator 106. Now, the operation of 
the supplemental information embedding apparatus accord- 
ing to the present invention constructed as mentioned above 20 
will be explained in reference to the flowchart shown in FIG. 
6. 

First, a normal image capturing operation (normal image 
capturing mode) of the present supplemental information 
embedding apparatus will be explained. 

In the normal image capturing mode, the made designator 
106 designates to connect the terminal a to the terminal cl, 
and the terminal a to the terminal d of the switch 107, the 
terminal f to the terminal g of the switch 108, and the 3Q 
terminal i to the terminal k of the switch 109. 

Accordingly, the image data captured by the CCD 101 for 
capturing an image by op optoelectrically transducing the 
light from the image object are applied to the digital image 
data processor 102 (steps SI, S2), encoded, and converted 35 
into the JPEG compression data. The converting operation 
into the JPEG compression data will be explained in refer- 
ence to FIG. 5. Here, in the normal image capturing mode, 
the terminal p is being connected to the terminal n in the 
switch 207, and the terminal q is being connected to the 40 
terminals in the switch 208. 

In FIG, 5, the input image data input from the terminal cl 
are divided into 8x8 pixel blocks for instance, and input to 
the DCT (frequency converter) 201 for carrying out DCT 
(Discrete Cosine Transform). In the DCT processor 201, i.e., 45 
the frequency converter, the 8x8 pixel blocks are arranged in 
different order of the frequency ingredient. The rearranged 
blocks (DCT coefficient) are rounded off to the nearest 
integer after the coefficient corresponding to the matrix is 
divided by the quantization matrix table in the quantizer 202 50 
following the DCT processor 201. Accordingly, the first 
stage of the compression processing (non-reversible 
compression) is carried out (an amount of codes is reduced). 

Since the information of the matrix data quantized in the 
quantizer 202 concentrates in the region of the low fre- 55 
quency components in the horizontal and vertical direction 
of the DCT conversion coefficient, the quantization output, 
i.e., the two dimensional format by the matrix scanning is 
converted into the one dimensional format by the zigzag 
scanning in the scanner 203. The one dimensional format 60 
data by the zigzag scanning are output to the VLC processor 
205 via the terminal p and the terminal n of the switch 207, 

In the VLC processor 205, the quantization output is 
processed into the Huffman encoding by, e.g., looking up the 
Huffman table. The Huffman encoding is for reducing the 65 
amount of codes by assigning a short code to the data having 
a high incidence and a long code to the data having a low 



138 Bl 

8 

incidence. Accordingly, the second compression processing 
(reversible compression) is carried out, so that the com- 
pressed data of the blocks are obtained. 

By carrying out the operation mentioned above on all of 
the 8x8 pixel blocks of input image data, a fully compressed 
data is obtained (step S3). The obtained compressed data are 
output to the terminal f of the switch 108, as shown in FIG. 
4, then output (recorded) to the external memory such as the 
memory card via the terminal g (step S4). Hereinafter, the 
format of the fully compressed data is referred to as a first 
format. 

Now, a digital watermark storing mode for storing the 
digital watermark such as the copyright data to the digital 
watermark memory 105 will be explained. Here, the digital 
watermark storing mode is classified into a first digital 
watermark storing mode for storing the image data captured 
by the image capturing device as the digital watermark to the 
digital watermark memory 105, and a second digital water- 
mark storing mode for storing the data stored in the external 
memory 103 such as a PC memory card or an IC memory 
card to the digital watermark memory 105 as the digital 
watermark. 

First, the first digital watermark storing mode will be 
explained. In the first digital watermark storing mode, the 
mode designator 106 designates to connect the terminal a to 
the terminal a and the terminal a to the terminal d of the 
switch 107, connect the terminal f to the terminal h of the 
switch 108, and connect the terminal i to the terminal k of 
the switch 109. 

The digital watermark is produced by capturing the 
character, logo mark or trademark which will be the copy- 
right data written on the paper, or capturing a portrait of the 
copyright owner (steps SI, S5, 513, 514, and S15). Then, the 
image data captured as mentioned above, i.e., the image data 
captured by the image capturing device such as the CCD 101 
is converted into the reduced-size picture in the digital 
watermark processor 104, and the reduced-size image data 
is carried out the JPEG compression (step S 16). 

To record the image compression data to the watermark 
information memory 105 as the digital watermark, the 
digital watermark is recorded (step S17). Hereinafter, the 
format of the image compression data which is converted 
into the reduced-size image data and processed through the 
JPEG compression will be referred as a second format. 

Now, the second digital watermark storing mode will be 
explained. In the second digital watermark storing mode, the 
mode designator 106 designates the terminal a to the termi- 
nal b and the terminal a to the terminal d in the switch 107, 
the terminal f to the terminal h in the switch 108, and the 
terminal i to the terminal j in the switch 109. 

The digital watermark is obtained by selecting the copy- 
right data stored in the external memory 103 (steps SI, S5, 
513, 514, and S18). When the selected data are the image 
data, the image data read out from the external memory 103 
are converted to the reduced-size picture, and the reduced- 
size picture is carried out the JPEG compression in the 
digital watermark processor 104, as same as the first digital 
watermark storing mode (step S 16). According to record 
image compression data in the digital watermark memory 
105 as the digital watermark, the digital watermark is 
recorded (step S 17), 

On the other hand, when the selected data are text data 
indicating the copyright and using the character fine "(C) 
Copyright TOSHIBA" as the copyright data, the character 
line is converted to ASCII code and recorded in the water- 
mark information memory 105 as the text data itself, so that 
the digital watermark is recorded. 
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By the way, in the explanation mentioned above, when the embedding supplemental information. Accordingly, if the 

image data are used as the digital watermark, the captured supplemental information is embedded into the image data 

color image data are changed simply to the reduced-size by one byte to one block of the image data, and the end code 

picture. However, if only the copyright is shown, a reduced- is set to {l}, data to be embedded into the one dimensional 

size picture may be produced by extracting only the hum- 5 f ormat win take a combination as follows. Since the data to 

nance signal and then carrying out the JPEG compression. be actually embedded ^ be 2 bytes (-supplemental 

Accordingly, the data amount of the digital watermark will information+ending code) for 2 pixels, the combination will 

be compressed more because of the lack of the chrominance be r» reseated bv 
data, so as to eventually reduce the data amount of the 

supplemental information. That is, if the supplemental infor- 3Q 

mation amount is less, the reduction of image quality will be r 81 } r E1 i r 51 | r 21 | | 91 i | 31 | r 61 | 

kept in minimum when the picture embedded with the ' ' ' * ' ' 

digital watermark is decoded. M exam ple of the one dimensional format embedded 

Now, the digital watermark embedding mode for embed- with the supplemental information {8, 1} in place of the 

ding the digital watermark such as the copyright data into the block 1 in the image data will be described hereinafter. Here, 

image data will be explained. Here, the digital watermark the where lhe data are embedded will be previously 

embedding mode is classified mto a first digital watermark defined M m ^ of normal ictures has a relatively 

embedding mode for embedding the digital watermark into smaU in ^ horizontal ^ vertical M h fr enc > 

the picture captured by the image capturing device i.e., the ioi £ ^ ^ ^ ^ ^ and ^ ^ * 

CCD 101 in a realtime manner and a second digital water- • *u \ *u *• *■ az • . • • a .l 

„ t . , r « jj* ,i j » • . i . - n is, the larger the quantization coefficient is assigned, the 

mark embedding mode for embedding the digital watermark 20 u • ♦ i a i w u r cTu 

■ ♦ *u mrr • • j . f . , horizontal and vertical high frequency regions of the quan- 

mto the JPEG compression image data stored in the memory # » • j- • ir * j . i . L 

u no F , b ; tization output, i.e., the one dimensional format data by the 

such as the PC memory card. . . , . . . f 4 , . 7 . 

„ . - . / zigzag scanning has a high possibility of taking a succession 

First, the first digital watermark embedding mode will be of i c ? a ^To-run. In this example, when the first part 

explained. In the first digital watermark embedding mode, of the before me E0B wiU be taken> the bk)ck x [s 

the mode designator 106 designates to connect the terminal 23 iven ^ f o ]] 0WS 
a to the terminal cl and the terminal a and the terminal c2 
of the switch 107, the terminal f to the terminal g of the 

switch 108, and the terminal i to the terminal k of the switch block l-{24, i-{24, -9, 8, o, 6, 4, -2, o, 5, -i, 3, o, i, o, o, o, o, 

109. i, 6, i, o, . . . o/o, o, EOB}! 

As an example of digital watermark such as JPEG com- 30 TT . 

pression image data embedding/imbedding technique, there Hereinafter, the supplemental data is divided into blocks, 

is a technique as disclosed in the Japanese Patent Applica- and then the blocks m dispersively embedded into the 

tion H8-318223. Hereinafter, an example of the digital data durm § three sta S es of processing, i.e., the DCT 

watermark embedding/unbedding will be described by processing, the quantization processing, and the scanning 

incorporating the digital watermark embedding device dis- 35 Posing have been completed on the way of the JPEG 

closed in the Japanese patent application H8-318223 in the compression encoding to other blocks as the same way (step 
supplemental information embedding apparatus according 

to the present invention Then » the dl § ltal watermark is processed through the 

The image data captured by the CCD 101 for capturing an variable len ^ th en °° din S re ™^ in the ™G compres- 

image by op optoelectrically transducing the light from the 40 ?°f f 0CGSS camed ! ™ l m the WL ^ ™ S ! unt ^ lbe 

image object are applied to the digital image dala processor ^ compression data are completely embedded with the 

102 via the terminals a and cl of the switch 107 (steps SI, dlgltal watera f rk ( ste P S1 °); Here > P™ duced JPEG 

S5, S6 and S7). Referring now to FIG. 5, an operation of the com P res f on data ™ s < ored «> the exter ? al ^mory 103 

digital image data processor 102 will be explained. Here, in suc * f the mc ^l f rd ™ the terminal f and ^ termmal 

thismode,theterminalpisconnectedtotheterminaloinlhe 45 g ° f th * ™*? ^^Jf*' u 

switch 207, and the terminal q is connected to the terminal In the ™ G ' the data « anning processing are 

s in the switch 208, as shown in FIG. 5. non-reversible compression, while the data after the scan- 

t, . * . . 4 c_ 4 i 4 • i -i j ning processing are the reversible compression. By embed- 

Ine image data input from the terminal cl are processed A > fu i . i • *• *• j . * *i_ j * c ^ 

♦u u * e *u j- * *i_ F .™; dmg the supplemental mformation data to the data after the 

through part of the encodings, i.e., the DCT processing, the „ ■ " . iL , . , - c -n . 

5 f. ... • ' . . 50 scanning processing, the supplemental information will be 

quantization, and the scanning through the DCT processor ♦ j r *i i! *i_ 

2 M ' . J» to - M /.l. ^ . restored perfectly when the reversible compression process- 

201, the quantizer 202, and the scanner 203 (the first m j carried out 

compression) (step S8). Then, the one dimensional format \ T iU \ j. v , 4 , . , .„ 

* *r *i_ mi - i- j i . r Now, the second digital watermark embedding mode will 

output from the scanner 203 is supplied to the information Ko av J ■ A ¥ a a- * ^ * i u aa- 

L a a niA • * -1 j c .t. • . he explained. In the second digital watermark embedding 

embedder 204 via the terminals p and o of the switch 207. « „ A 4U A . ■ . . . , t t , 6 

— . - . , , , . 55 mode, the mode designator 106 designs to connect the 

The information embedder 204 receives the one dimen- terminal a t0 the terminal b and the terminal a t0 the terminal 

sional format on the way of the JPEG compressing ^ in the switch 107> the terminal f to the terminal in the 

operation and the digital watermark stored in the digital switch wg and the terminal j to the terminal k in the switch 

watermark memory 105, as shown in FIG. 4, through the m Furtherf the terminal ^ ^^eded to the terminal m 

termmals a and c2 of the switch 107. 6Q ^ the switch ^ and the tefminal q fa Mected tQ lhe 

As an example, scanned data corresponding to block 1 terminal r in the switch 208, as shown in FIG. 5. 
with 8x8 pixels are presented by; ^ JPEG compression data to which any digital water- 
block 1={24, -9, 8, 0, 6, 4, -2, 0, 5, -1, 1, 0, 1, 0, 0,. 0, mark stored in the external memory 103 such as the memory 
0, 1, 0, 0, 0, ... 0, 0, 0, EOB} and data to be embedded has card is not yet embedded are processed through the variable 
a format {8E529363} will be described. 65 length encoding in the VLD processor 206 via the terminal 
At this time, it is possible to prevent the reduction of q and the terminal r of the switch 208 and supplied to the 
image quality during JPEG expansion by dispersively information embedder 204 (steps SI, S5, S6, 511, and S 12). 
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Here, the variable length encoded data has similar block data 
as explained in the first digital watermark embedding mode. 

On the other hand, the digital watermark is applied to the 
information embedder 204 through the terminals a and c2 of 
the switch 107. 5 

Then, similar to the first digital watermark embedding 
mode, the digital watermark obtained by the digital water- 
mark memory 105 is embedded into the data variable length 
decoded by the VLD processor 206, which is input into the 
information embedder 204 (step S9). The information 
embedded data are then variable length encoded in the VLC 
processor 205, so that a JPEG compression data embedded 
with the digital watermark is obtained (step S30). The 
produced JPEG compression data are stored in the external 
memory 103 such as the memory card via the terminal f and 
the terminal g of the switch 108 (step S4). 15 

Even if the JPEG compression data to which the digital 
watermark obtained as mentioned above is embedded are 
carried out the JPEG expansion based on the JPEG standard 
and displayed on the monitor by the software for the image 
display, the embedded digital watermark are simply shown 20 
as a noise, so that the image quality is simply reduced a little. 

To unbed the digital watermark embedded into the image 
data so as to display the copyright data, an unbedding 
apparatus and its method disclosed in the Japanese patent 
application H08-3 18223 can be available. FIG. 7 is a block 25 
diagram showing one embodiment of the unbedding appa- 
ratus. 

In FIG. 7, the JPEG compression data are supplied to a 
terminal g from the external memory 103 such as the 
memory card (not shown). The JPEG compression data are 30 
supplied to a VLD processor 301 for carrying out the 
variable length decoding. The output of the VLD processor 
301 is supplied to a supplemental information unbedder 302 
for inversely carrying out the operation of the information 
embedder 204. The supplemental information, e.g., the 35 
digital watermark unbedded by the supplemental informa- 
tion unbedder 302, is converted to the copyright data (the 
picture, text, logo mark or trademark representing the 
copyright) in a data converter 306, then displayed to the 
supplemental information (copyright data) display (not 40 
shown). 

On the other hand, the variable length decoded JPEG 
compression data after the supplemental information has 
beef removed therefrom are supplied to a reverse scanner 
303 for carrying out the opposite processing to the scanner 45 
203, a reverse quantizer 304 for carrying out processing 
opposite to that of the quantizer 202, and an IDCT processor 
305 for carrying out the opposite processing to the DCT 
processor 201 in the order, and then supplied to an image 
data display for displaying the main image data thereon. 50 

As described above, i.e., by carrying out a processing 
opposite to that carried out in the first digital watermark 
embedding mode, the digital watermark is unbedded. Here, 
the apparatus, as shown in FIG. 7, may be integrated with the 
apparatus as shown in FIG. 1. 55 

As mentioned above, the first aspect of the present inven- 
tion carries out the JPEG compression as the digital image 
processing in the digital image data processor 102. However, 
the data may be other digital still picture compression data 
or moving picture data operated by an MPEG standard, a 60 
standard for moving picture compression. Thus, the basic 
hardware thereof has a configuration similar to the configu- 
ration shown in FIG. 4. In this case, the internal arrangement 
of the digital image data processor 102 will correspond to 
the encoding algorithm. 65 

Further, any data may be stored in the digital watermark 
memory 105 if the data can show the presence of copyright 
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protection and can be converted to the digital data. That is, 
the data may be, for example, audio data. In case of audio 
data, the audio data are stored in the memory for audio via 
the external memory 103. 

Furthermore, the embodiment of the present invention 
explains the case of using the PC memory card as the 
memory. However, the memory may be the disc medium or 
tape medium if it is ready to write and read the data thereto 
and therefrom. In this case, it is the matter of course to use 
a drive corresponding to each medium. 

Hereinafter, second aspects of the present invention will 
be described. 

FIG. 8 is a block diagram showing the second aspect of 
the supplemental information embedding apparatus of the 
present invention. 

As shown in FIG. 8, the supplemental information embed- 
ding apparatus is comprised of a DCT processor 1 for 
orthogonal transforming to decompose an original image 
data which was divided into 8x8 pixel blocks to frequency 
components by the DCT (Discrete Cosine Transform) 
processing, a scanner 2 for, e.g., zigzag scanning or alternate 
scanning the blocks (DCT coefficients) obtained by the DCT 
processing to convert them in a two dimensional format to 
a one dimensional format, a quantizer 3 for quantizing each 
of DCT coefficients of the blocks transformed to one dimen- 
sional format by using coefficients stored in a quantization 
matrix table, an information embedder 4 for embedding 
supplemental information 6 such as character information or 
ID information for identifying individuals, etc. into quan- 
tized block data according to predetermined requirements, 
and a VLC (Variable Length Coding) processor 5 for car- 
rying out a reversible compression for block data embedded 
with the information by looking up a Huffman table. 

Now, the operations carried out by the supplemental 
information embedding apparatus will be described. FIG. 9 
is a flowchart showing the operations carried out by the 
supplemental information embedding apparatus. 

Original images for one frame (640x480 pixels) that are 
input are divided into 8x8 pixel blocks (step S22), and then 
input to the DCT processor 1 (step S23). The DCT processor 

1 carries out the DCT processing of input blocks (step S24). 
8x8 pixel blocks data are rearranged to frequency compo- 
nents by this DCT processing. The block data that are 
rearranged to frequency components are input to the scanner 

2 and converted into one dimensional format from two 
dimensional format (step S25). The block data processed the 
scanning are quantized in the quantizer 3 using the quanti- 
zation matrix table (step S26). 

FIG. 10 shows examples of the one dimensional format of 
each of the quantized blocks. Here, the EOB (End of Block) 
is a code indicating the end of block, while the section 
shown with three dots is a zero-run section. In the informa- 
tion embedder 4, supplemental information 6 is embedded 
into such quantized blocks according to the requirements 
shown below: 

(1) (Supplemental Inform ation+End Code) are embedded 
in place of the zero-run section immediately preceding 
the EOB. The end code is a code for indicating the end 
position of the supplemental information. 

(2) End code should be {l}. 

(3) (Supplemental Inform ation+End Code) are embedded 
in the low frequency region (=the zero-run start 
portion) of the zero-run section immediately preceding 
the EOB by N pixels (for instance, N bytes by assuming 
1 byte per pixel). Here, N is an optional set figure. 

(4) If the count of zero-runs immediately preceding the 
EOB is not larger than a set figure M in the state of 
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supplemental information being embedded, supple- 
mental information will not be embedded. 

According to the above requirements (1) and (2), supple- 
mental information can be embedded without impairment of 
the original image compression data. That is, original image 5 
compression data (i.e., image compression data not embed- 
ded with supplemental information) can be restored by 
uniquely replacing a code section due to the supplemental 
information with a zero-run in data decoding operation. 
Further, according a to the requirement (3), it is possible to 
make a compression ratio as high as possible by the revers- 
ible compression (Huffman encoding). That is, if supple- 
mental information is embedded in the middle portion or the 
end portion of the zero-run, the length of the zero-run 
section immediately preceding the EOB becomes short and 
the compression ratio drops. Further, according to the 15 
requirement (4), it becomes possible to discriminate a block 
with and a block without supplemental information embed- 
ded therein at the time of decoding data. If N and M are set 
at "2", the underlined section of each block becomes a 
section into which (Supplemental Information+End 10 20 
Code) can be embedded according to the requirements (1) 
and (3), as shown in FIG. 11. For instance, as the block 4 has 
only one zero-run section immediately preceding the EOB, 
supplemental information cannot be embedded according to 
the requirement (4). "i" in step S27 in the flowchart of FIG. 2 s 
9 represents the count of the zero-run length immediately 
preceding the EOB, while "n" represents N+M (=2+2). 

When embedding a Chinese letter name (i.e., supplemen- 
tal information) such as " W into these blocks, the Chinese 
letter name 'W is comprised " W"=8E52 and "ST-9363 in 30 
the shifted-JIS code, and therefore the supplemental infor- 
mation has a code sequence of {8E529363}. 

According to the above requirements (2) and (3), data to 
be in place of each of the above blocks processed the 
scanning will have combinations as shown below. 35 

Data to be embedded are 2 byte data for 2 pixels 
("■Supplemental In formation + End Code) and becomes {8, 
1}, {E, 1}, {5, 1}, {2, 1}, {9, 1}, {3, 1}, {6, 1}, {3, 1}. FIG. 
12 shows the resulting data which is embedded with the 2 
byte data (Supplemental information+End Code). 40 

After obtaining image compression data embedded with 
the supplemental information 6 in the information embedder 
4, the Huffman encoding (reversible compression) is 
executed for this image compression data in the VLC 
processor 5 (step S30). 45 

As described above, because the supplemental informa- 
tion has been embedded between the non-reversible data 
compression by the orthogonal transforming operation and 
the reversible data compression by the Huffman encoding, 
supplemental information is no longer lost in the subsequent 50 
compression. 

It is also possible to embed other supplemental 
information, for instance, ID information, codes, etc. for 
identifying individuals by executing the similar operation 
for all blocks in one frame, without impairment of original 55 
image compression data. If ID information or codes for 
identifying individuals are embedded into image compres- 
sion data, as described above, it becomes possible to easily 
find any illegal copy of digital image data by restoring such 
supplemental information. 60 

In case of the VGA (Video Graphics Array) screen (640x 
480 pixels), as there are 4,800 blocks (=640x480/(8x8)), it 
is possible to includes 1,200 characters (=4,800/4) of the 
shifted JIS code at maximum in image compression data for 
one frame. 65 

Further, in this embodiment supplemental information 
and the end code were embedded in place of the zero-run 



,138 Bl 

14 

section immediately preceding the EOB. However, when 
image data are compressed using a succession of another 
figure, e.g., a succession of "1", supplemental information 
and the end code may be embedded in place of the run of the 
figure. Further, supplemental information and the end code 
were embedded in place of the first zero-run section, but a 
combination of start code, supplemental information and 
end code may be embedded in place of any mid portion of 
the zero-run. 

Further, although the image data was divided into 8x8 
pixel blocks, it 5 may be divided into other size of pixel 
blocks, e.g., 16x8 pixel blocks. 

The compression data thus produced by embedding 
supplemental information into the original image data are 
recorded on a recording medium such as a ROM, an IC 
memory, a magnetic disc, a floppy disc, a CD, a DVD, etc. 

Now, an image decoder for decoding image compression 
data obtained by the image compressor, as shown in FIG. 8, 
will be described. 

FIG. 13 is a block diagram showing a part of such an 
image decoder. As shown in FIG. 13, the image decoder is 
comprised of a VLD processor U for variable length decod- 
ing of image compression data by looking up a Huffman 
table, a supplemental information unbedder 12 for unbed- 
ding the supplemental information from data obtained 
through the VLD processing, a data converter 13 for con- 
verting the supplemental information unbedded by the 
supplemental information unbedder 12 into data to be dis- 
played by accumulating them for every one frame unit, a 
reverse quantizer 14 for reverse quantizing data restored by 
replacing the data unbedded by the supplemental informa- 
tion unbedder 12, i.e., the combination of the supplemental 
information and the end code with original zero-run, a 
reverse scanner 15 for reverse scanning the reverse quan- 
tized data to reconvert them in one dimensional format to the 
two dimensional format and an IDCT processor 16 for 
inverse DCT processing the blocks processed the reverse 
scanning. 

Now, an image decoding carried out by the image decoder 
will be described. Shown in FIG. 14 is a flowchart showing 
the steps of such an image decoding. 

First, the variable length decoding is executed in the VLD 
processor 11 for one block of image compression data of 
input image compression data for one frame (step S73). As 
a result, block data immediately preceding the variable 
length encoding in the data compression, as shown in FIG. 
12, are obtained. The supplemental information unbedder 12 
unbeds supplemental information from the block data as 
follows according to the above-described requirements (1)- 
(4). 

First, the length of a zero-run immediately preceding the 
EOB of variable length decoded block data is counted (step 
S74). Then, blocks embedded with and without supplemen- 
tal information are discriminated according to the count and 
the requirement (4), 

That is, if the length of the zero-run immediately preced- 
ing the EOB is not longer than a set figure M, it is found that 
no supplemental information is embedded according to the 
requirement (4). Then, the count of zero-runs is compared 
with the set figure M, and if i^M, it is possible to judge that 
supplemental information is embedded, while in other cases 
there is no supplemental information embedded (step SIS). 
For instance, in case of Block 4 shown in FIG. 12, i equals 
to 1 and M equals to 2. And thus, it is seen that there is no 
supplemental information embedded therein. When supple- 
mental information is embedded, as it is known that supple- 
mental information+end code "1" for N (e.g., 2) pixels have 
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been embedded according to the requirement (3), it is 
possible to extract data for 2 pixels in front of a zero-run 
immediately preceding the EOB as supplemental informa- 
tion and the end code from each block. 

In other words, {8, 1}, {E, 1}, {5, 1}, {2, 1}, {9, 1}, {3, 5 
1}, {6, 1}, {3, 1} are extracted from each block. 

Further, as the section embedded with supplemental infor- 
mation at the time of compression was originally a zero-run, 
the original image compression data, as shown in FIG. 10, 
can be restored by replacing the section with a correspond- 10 
ing count of zeros (step S76). 

When the image compression data is reverse quantized in 
the reverse quantizer 14 by looking up the Huffman table 
(step S77), and then converted into two dimensional format 
in the reverse scanner 15 (step S78) and further, are inverse 15 
DCT processed in the IDCT processor 16 (step S79), blocks 
that are close to its original image. 

On the other hand, the supplemental information unbed- 
der 12 separates information extracted from blocks into 
supplemental information and the end code according to the 2 q 
requirements (1) and (2), and after removing "1" that is the 
end code, the supplemental information only is sent to the 
data converter 13. The data converter 13 accumulates 
supplemental information for whole blocks of one frame 
(step S82), and restores original supplemental information 2 $ 
such as image, audio or character data. The original supple- 
mental information is then output to corresponding periph- 
eral equipment (step S83). 

That is, as it is known that supplemental information is {8, 
E, 5, 2, 9, 3, 6, 3} and character information, the Chinese 30 

letter name "jUpT' is obtained by the data conversion circuit 
13. 

Further, although the present invention applied to the 
JPEG to implement the quantization after the scanning at the 
time of image compression has been described in this 35 
embodiment, the present invention is also applicable simi- 
larly to such a compression system using frequency conver- 
sion and quantization as the MPEG, etc. which processes the 
scanning after quantization. 

According to the first aspect of the invention, the digital 40 
watermark to be protected by the copyright can be easily 
embedded into the captured image or video data by a 
portable image capturing apparatus such as the electronic 
still camera or the digital video camera without using a 
large-scale system. 45 

According to the second aspect of the invention, it 
becomes unnecessary to store supplemental information in a 
separate file and also it is able to decrease the reduction of 
reproduced image quality caused by embedding the supple- 
mental information into the image compression data in a 50 
negligible order. That is, as supplemental information are 
embedded in place of the zero-run section of a block, that is, 
at the high frequency side, even when image compression 
data with supplemental information embedded therein are 
directly decoded by a conventional image decoder, noise by 55 
supplemental information is visually recognized and a repro- 
duced image without any problem for practical use is 
obtained. 

According to the other aspect of the invention, it becomes 
possible to minimize the drop of data compression ratio 60 
resulting from the reversible compression by the Huffman 
encoding, etc. 

According to the other aspect of invention, the image 
decoder is able to reproduce image, audio or character data 
by unbedding supplemental information from the image 65 
compression data processed by the supplemental informa- 
tion embedding apparatus according, to the present 



invention, and also able to completely restore original image 
compression data thus keeping the quality of the original 
image by replacing the supplemental information with a 
zero- run. 

As supplemental information corresponding to images for 
one frame are embedded in place of the head of the zero-run 
section immediately preceding the EOB by dispersing them 
into plural blocks, it becomes possible to increase the 
volume of supplemental information for one frame while 
minimizing the drop of data compression ratio by the 
reversible compression by the Huffman encoding, etc. 

As described above, the present invention can provide an 
extremely preferable apparatus and method for embedding/ 
unbedding supplemental information and record medium 
therefor. 

While there have been illustrated and described what are 
at present considered to be preferred embodiments of the 
present invention, it will be understood by those skilled in 
the art that various changes and modifications may be made, 
and equivalents may be substituted for elements thereof 
without departing from the true scope of the present inven- 
tion. In addition, many modifications may be made to adapt 
a particular situation or material to the teaching of the 
present invention without departing from the central scope 
thereof. Therefor, it is intended that the present invention not 
be limited to the particular embodiment disclosed as the best 
mode contemplated for carrying out the present invention, 
but that the present invention includes all embodiments 
falling within the scope of the appended claims. 

The foregoing description and the drawings are regarded 
by the applicant as including a variety of individually 
inventive concepts, some of which may lie partially or 
wholly outside the scope of some or all of the following 
claims. The fact that the applicant has chosen at the time of 
filing of the present application to restrict the claimed scope 
of protection in accordance with the following claims is not 
to be taken as a disclaimer or alternative inventive concepts 
that are included in the contents of the application and could 
be defined by claims differing in scope from the following 
claims, which different claims may be adopted subsequently 
during prosecution, for example, for the purposes of a 
divisional application. 

What is claimed is: 

1. A supplemental information embedding apparatus, 
comprising: 

a device for capturing a first image data; 

a first memory for storing a second image data indicating 

the presence of copyright protection; 
a first encoder for encoding the first image data obtained 

by the image capturing device to a first format digital 

image data by n stages of processing (n is a positive 

integer greater than or equal to 2); 
a second memory for storing the first format digital image 

data produced in the first encoder; 
a second encoder for encoding the image data indicating 

a copyright data supplied from the image capturing 

device or the first memory to a second format digital 

watermark by i stages of processing (i is a positive 

integer less than n); 
a third memory for storing the second format digital 

watermark encoded by the second encoder; 
a decoder for decoding the first format digital image data 

stored in the second memory up to a prescribed stage of 

decoding; 

a divider for dividing the second format digital watermark 
stored in the third memory into predetermined format 
blocks; 
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an embedder for dispersively embedding the blocks of the 
digital watermark into the image data encoded up to i 
stage by the decoder; 

a third encoder for encoding the image data obtained by 
the embedder by exerting operations on or after j stage 5 
(=n-l) of encoding so as to produce the first format 
digital image data; and 

a fourth memory for storing the first format digital image 
data encoded by the third encoder. 

2. A supplemental information embedding method, com- 10 
prising: 

receiving an image data from an image capturing device; 

carrying out a first encoding operation for producing 
digital image data in a first format by n stages of 15 
processing (n is a positive integer greater than or equal 
to 2) for the image data captured by the image capturing 
device; 

carrying out a second encoding operation for producing 
digital watermark in a second format by i stages of 20 
processing (i is a positive integer less than n) for a 
second image data indicating a presence of copyright 
Protection supplied from the image capturing device or 
a first memory storing a digital image data indicating 
the presence of copyright protection; 25 

decoding the first format digital image data stored in a 
second memory up to a prescribed stage; 

dividing the second format digital watermark stored in a 
digital watermark memory into predetermined format 
blocks; 30 

dispersively embedding the blocks of the digital water- 
mark into the image data encoded up to i stage by the 
first encoding operation or the image data decoded up 
to i stage by the decoding operation; ^ 

carrying out a third encoding operation on or after j stages 
of processing (j=n-i) to the image data obtained in the 
embedding of the digital watermark so as to reproduce 
the second format digital image data; and 

storing the first format digital image data encoded in the 40 
third stage encoding operation in a memory. 

3. An image processing method as claimed in the claim 2, 
further comprising: 

receiving another image data from the image capturing 

device; 45 
encoding the other image data to the second format data 

so as to produce the digital watermark indicating the 

presence of copyright protection; and 
storing the digital watermark into the digital watermark 

memory. 50 

4. An image processing method as claimed in claim 2, 
further comprising: 

reading out the digital image data indicating the presence 
of copyright protection from the first memory; 

encoding the digital image data to the second format so as 
to produce the digital watermark indicating the pres- 
ence of copyright protection; and 

storing the digital watermark into the digital watermark 
memory. 6Q 

5. An image processing method as claimed in claim 2, 
further comprising: 

receiving another image data from the image capturing 
device; 

carrying out encoding operations up to i stage (i is a 65 
positive integer less than n), to the other image data 
captured by the image capturing device; 
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dividing the second format digital watermark stored in the 
digital watermark memory into a specific format data; 

dispersively embedding the blocks of the digital water- 
mark into the image data encoded up to i stage; 

carrying out the encoding operations on or after j stage 
(j=n-l) to the data embedded with digital watermark so 
as to reproduce the first format digital image; and 

storing the first format image data into a third memory, 

6. An image processing method as claimed in claim 2, 
further comprising: 

reading out the first format encoded image data stored in 
the second memory; 

carrying out decoding operations up to i stage for the 
image data read out from the second memory; 

dividing the second format digital watermark stored in the 
digital watermark memory into blocks; 

dispersively embedding the blocks of the digital water- 
mark so as to produce a specific format image data; 

carrying out encoding operations on or after j stage 
(j=n-l) to specific format image data embedded with 
digital watermark so as to reproduce the first format 
digital image data; and storing the first format digital 
image data into the third memory. 

7. A supplemental information embedding apparatus, 
comprising: 

an orthogonal transformer for orthogonal transforming 

divided blocks of image data; 
a quantizer for quantizing the divided blocks of image 

data after the orthogonal transform; and 
an embedder for embedding supplemental information in 

place of at least a part of a zero-run section immediately 

preceding an EOB (End of Block) in each of the 

quantized blocks of image data. 

8. A supplemental information embedding apparatus, 
comprising: 

an orthogonal transformer for orthogonal transforming 

divided blocks of image data; 
a quantizer for quantizing the divided blocks of image 

data after the orthogonal transform; and 
an embedder for embedding supplemental information in 

place of the head of a zero-run section immediately 

preceding an EOB (End of Block) in each of the 

quantized blocks of image data. 

9. A supplemental information embedding apparatus, 
comprising: 

an orthogonal transformer for orthogonal transforming 
divided blocks of image data; 

a quantizer for quantizing the divided blocks of image 
data after the orthogonal transform; and 

a divider for dividing supplemental information corre- 
sponding to one frame of the image data into a plurality 
of blocks; and 

an embedder for dispersively embedding the divided 
blocks of supplemental information into the image data 
in place of the head of a zero run section immediately 
preceding an EOB (End of Blocks) of the quantized 
blocks of image data. 

10. A supplemental information embedding apparatus as 
claimed in any one of claims 7 to 9, further comprising: 

a reversible compressor for reversible compressing the 
blocks of image data embedded with the supplemental 
information. 

11. An image decoder for decoding image compression 
data wherein supplemental information is embedded in place 
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of at least a part of zero-run section immediately preceding 
an EOB (End of Block), which is obtained by dividing 
image data, orthogonal transforming the divided blocks of 
image data and quantizing the orthogonal transformed 
blocks of image data, comprising: 5 
an unbedder for unbedding the supplemental information 

from the image compression data; 
a memory for storing the imbedded supplemental infor- 
mation; a replacer for replacing the supplemental infor- 
mation embedded in the image compression data with 10 
a zero-run; 

a reverse quantizer for reverse quantizing the zero- 
replaced image compression data; and 

a reverse orthogonal transformer for reverse orthogonal ^ 
transforming the reverse quantized image compression 
data. 

12. An image decoder for decoding reversible compressed 
image data wherein supplemental information is embedded 

in place of at least a part of a zero-run section immediately 2 o 
preceding an EOB (End of Block) obtained by dividing 
image data, orthogonal transforming the divided blocks of 
image data and quantizing the orthogonal transformed 
blocks of image data, comprising: 

a reversible decoder for reversible decoding the image 25 

compression data; 
an unbedder for unbedding the supplemental information 
from the reversible decoded image compression data; 
a memory for storing the unbedded supplemental infor- 
mation; 30 

a replacer for replacing the supplemental information 
embedded in the image compression data with a zero- 
run; 

a reverse quantizer for reverse quantizing the zero- 35 

replaced image compression data; and 
a reverse orthogonal transformer for reverse orthogonal 

transforming the reverse quantized image compression 

data. 

13. An image encoding method comprising: 40 
orthogonal transforming divided blocks of image data; 
quantizing the divided blocks of image data after the 

orthogonal transforming operation; and 
embedding supplemental information into at least a part 45 
of a zero-run section immediately preceding an EOB 
(End of Block) in each of the quantized blocks. 

14. An image encoding method comprising: 
orthogonal transforming divided blocks of image data; 
quantizing the divided blocks of image data after the 50 

orthogonal transforming operation; and 
embedding supplemental information in place of the head 
of a zero-run section immediately preceding an EOB 
(End of Block) in each of the quantized block. 

15. An image encoding method comprising: 55 
orthogonal transforming divided blocks of image data; 
quantizing the divided blocks of image data after the 

orthogonal transforming operation; 

dividing supplemental information corresponding to one 60 
frame of image data into a plurality of blocks; and 

dispersively embedding the divided blocks of supplemen- 
tal information into the image data in place of the head 
of a zero-run section immediately preceding an EOB 
(End of Blocks) of the quantized blocks of image data, 65 

16. A method for decoding image data wherein supple- 
mental information is embedded in place of at least a part of 
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a zero-run section immediately preceding an EOB (End of 
Block), which is obtained by dividing image data, orthogo- 
nal transforming the divided blocks of image data and 
quantizing the orthogonal transformed blocks of image data, 
comprising: 

unbedding the supplemental information from the image 

compression data; 
storing the unbedded supplemental information; 
replacing the supplemental information embedded in the 

image compression data with a zero-run; 
reverse quantizing the zero-replaced image compression 

data; and 

reverse orthogonal transforming the reverse quantized 
image compression data. 

17. A method for decoding reversibly compressed image 
data wherein supplemental information is embedded in place 
of at least a part of the zero run section of immediately 
preceding an EOB (End of Block), which is obtained by 
dividing image data, orthogonal transforming the divided 
blocks of image data and quantizing the orthogonal trans- 
formed blocks of image data, comprising: 

reversible decoding the image compression data; 

unbedding the supplemental information from the revers- 
ible decoded image compression data; 

storing the unbedded supplemental information; 

replacing the supplemental information embedded in the 
image compression data with a zero-run; 

reverse quantizing the zero-replaced image compression 
data; and 

reverse orthogonal transforming the reverse quantized 
image compression data. 

18. A supplemental information embedding apparatus, 
comprising: 

means for capturing a first image data; 

first storing means for storing a second image data indi- 
cating the presence of copyright protection; 

first encoding means for encoding the first image data 
obtained by the image capturing means to a first format 
digital image data by n stages of processing (n is a 
positive integer greater than or equal to 2); 

second storing means for storing the first format digital 
image data produced in the first encoding means; 

second encoding means for encoding the image data 
indicating a copyright data supplied from the image 
capturing means or the first storing means to a second 
format digital watermark by i stages of processing (i is 
a positive integer less than n); 

third storing means for storing the encoded second format 
digital watermark encoded by the second encoding 
means; 

means for decoding the first format digital image data 
stored in the second storing means up to a prescribed 
stage of decoding; 

means for dividing the second format digital watermark 
stored in the third storing means into predetermined 
format blocks; 

means for dispersively embedding the blocks of the 
digital watermark into the image data encoded up to i 
stage by the first encoding means or the image data 
decoded up to i stage by the decoding means; 

third encoding means for encoding the image data 
obtained by the digital watermark embedding means by 
exerting operations on or after j stage (j=n-l) of 
encoding so as to produce the first format digital image 
data; and 
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means for storing the first format digital image data 
encoded by the third encoding means. 

19. A supplemental information embedding apparatus, 
comprising: 

means for orthogonal transforming divided blocks of 5 
image data; 

means for quantizing the divided blocks of image data 
after the orthogonal transforming operation; and 

means for embedding supplemental information in place 3Q 
of at least a part of a zero-run section immediately 
preceding an EOB (End of Block) in each of the 
quantized blocks of image data. 

20. A supplemental information embedding apparatus, 
comprising: is 

means for orthogonal transforming divided blocks of 
image data; 

means for quantizing the divided blocks of image data 
after the orthogonal transforming operation; and 

means for embedding supplemental information in place 20 
of the head of a zero-run section immediately preceding 
an EOB (End of Block) in each of the quantized blocks 
of image data. 

21. A supplemental information embedding apparatus, 
comprising: 25 

means for orthogonal transforming divided blocks of 
image data; 

means for quantizing the divided blocks of image data 
after the orthogonal transforming operation; 3Q 

means for dividing supplemental information correspond- 
ing to one frame of the image data into a plurality of 
blocks; and 

means for dispersively embedding the divided blocks of 
supplemental information into the image data in place 35 
of the head of a zero-run section immediately preceding 
an EOB (End of Blocks) of the quantized blocks of 
image data. 

22. A supplemental information embedding apparatus as 
claimed in any one of claims 19 to 21, further comprising: 40 

means for reversible compressing the blocks of image 
data embedded with the supplemental information. 



23. An image decoder for decoding image compression 
data wherein supplemental information is embedded in place 
of at least a part of zero-run section immediately preceding 
an EOB (End of Block), which is obtained by dividing 
image data, orthogonal transforming the divided blocks of 
image data and quantizing the orthogonal transformed 
blocks of image data, comprising: 

means for imbedding the supplemental information from 
the image compression data; 

means for storing the unbedded supplemental informa- 
tion; 

means for replacing the supplemental information embed- 
ded in the image compression data with a zero-run; 

means for reverse quantizing the zero replaced image 
compression data; and 

means for reverse orthogonal transforming the reverse 
quantized image compression data. 

24. An image decoder for decoding reversible compressed 
image data wherein supplemental information is embedded 
in place of at least a part of a zero -run section immediately 
preceding an EOB (End of Block) obtained by dividing 
image data, orthogonal transforming the divided blocks of 
image data and quantizing the orthogonal transformed 
blocks of image data, comprising: 

means for reversible decoding the image compression 
data; 

means for imbedding the supplemental information from 
the reversible decoded image compression data; 

means for storing the unbedded supplemental informa- 
tion; 

means for replacing the supplemental information embed- 
ded in the image compression data with a zero-run; 

means for reverse quantizing the zero replaced image 
compression data; and 

means for reverse orthogonal transforming the reverse 
quantized image compression data. 
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